Citation Counting, Citation Ranking, and h-Index of Human-Computer Interaction Researchers: A Comparison between Scopus and Web of Science

This study examines the differences between Scopus and Web of Science in the citation counting, citation ranking, and h-index of 22 top human-computer interaction (HCI) researchers from EQUATOR—a large British Interdisciplinary Research Collaboration project. Results indicate that Scopus provides significantly more coverage of HCI literature than Web of Science, primarily due to coverage of relevant ACM and IEEE peer-reviewed conference proceedings. No significant differences exist between the two databases if citations in journals only are compared. Although broader coverage of the literature does not significantly alter the relative citation ranking of individual researchers, Scopus helps distinguish between the researchers in a more nuanced fashion than Web of Science in both citation counting and h-index. Scopus also generates significantly different maps of citation networks of individual scholars than those generated by Web of Science. The study also presents a comparison of h-index scores based on Google Scholar with those based on the union of Scopus and Web of Science. The study concludes that Scopus can be used as a sole data source for citation-based research and evaluation in HCI, especially when citations in conference proceedings are sought, and that h scores should be manually calculated instead of relying on system calculations. INTRODUCTION Citation analysis—i.e., the analysis of data derived from references cited in footnotes or bibliographies of scholarly publications—is a powerful and popular method of examining and mapping the intellectual impact of scientists, projects, journals, disciplines, and nations (Borgman, 1990; Garfield, 1979; Meho, 2007; Moed, 2005). The method is increasingly being used by academic, research, and federal institutions in several countries worldwide for research policy making, visualization of scholarly networks, and monitoring of scientific developments, as well as for promotion, tenure, hiring, salary raise, and grants decisions, among others (see Borgman & Furner, 2002; Cronin, 1996; Small, 1999; Warner, 2000; Weingart, 2005; White & McCain, 1997, 1998). Indeed, several governments have been using or are considering using citation analysis and other bibliometrics measures/indicators to inform decisions Forthcoming: Journal of the American Society for Information Science and Technology